©-© 


Making an Impact 


Impact of providing information 
to parents in Texas about 
the role of Algebra Il in 
college admission 


Ginger Stoker 
Lynn Mellor 
American Institutes for Research 


Key findings 


This randomized controlled trial examined the impact on student completion of Algebra II 
in grade 11 of informing parents of high school students in Texas about the role of 
Algebra Il in college admission. Parents of students in treatment schools received an 
informational brochure about the role of Algebra II in college admission, and parents 
of students in control schools received a brochure outlining changes to high school 
graduation requirements. 
~ Students in treatment and control schools completed Algebra Il in grade 11 at a 
similar rate, after student and school characteristics were controlled for. 
Algebra Il completion rates in grade 11 did not differ between students in treatment 
and control schools with a high percentage of racial/ethnic minority students. 
Algebra Il completion rates in grade 11 did not differ between students in treatment 
and control schools with a high percentage of economically disadvantaged students. 
However, an exploratory analysis suggests that the effects may differ between 
students in low-income schools and students in schools that were not low-income 
schools. 
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Summary 


Prior to June 2013, Texas students could choose between two graduation plans, the default 
Recommended High School Plan and the Distinguished High School Plan, both of which 
required students to complete four courses each in English, math (including Algebra II), 
science, and social studies.! With the enactment of House Bill 5 (HB 5) in June 2013, 
those plans were replaced with the Foundation High School Program, which introduced 
flexibility into the course requirements for high school graduation, including a new default 
option that does not require Algebra II. 


The Foundation High School Program consists of three options: the basic 22-credit 
Foundation High School Program, which students need permission to select; the default 
26-credit option, which comprises the Foundation High School Program plus an endorse- 
ment (Foundation High School Program Plus Endorsement); and the distinguished level 
of achievement, which entails fulfilling the requirements of the Foundation High School 
Program Plus Endorsement and completing Algebra II. Only students who complete the 
science, technology, engineering, and math endorsement of the Foundation High School 
Program Plus Endorsement or a distinguished level of achievement are required to com- 
plete four courses each in English, science, and math, including Algebra II—the course 
requirements for admission to all public four-year colleges and universities in Texas. The 
students who entered grade 9 in the 2014/15 school year were the first cohort of students 
required to graduate under the Foundation High School Program. 


Without information on the alignment between the new high school graduation require- 
ments and college entrance requirements, parents may not be able to help their children 
make informed choices. This is true particularly for racial/ethnic minority and low-income 
parents, whose children are most likely to be affected by changes in graduation require- 
ments because they are the most likely to complete only the minimum graduation require- 
ments (Chaney, Burgdorf, & Atash, 1997; Domina & Saldana, 2011; Saw & Broda, 2012; 
Schiller & Muller, 2003). 


This study used a randomized controlled trial to examine the impact of informing 
parents in Texas about the role of Algebra II in college admission on student comple- 
tion of Algebra II in grade 11. A total of 109 schools, covering all 20 Education Service 
Center regions in Texas, participated in the study. Parents of students in the 54 treatment 
schools were mailed brochures containing information about the role of Algebra II in 
college admission and success, as well as information about the new high school gradua- 
tion options. Parents of students in the 55 control schools received brochures containing 
information about changes in the high school graduation requirements. 


The study, conducted in coordination with and on behalf of the Texas Education Agency, 
provides evidence to the agency and other states about the effect of informing parents 
about the role of Algebra II in college admission. 


Key findings include: 
e Students in treatment and control schools completed Algebra II in grade 11 at a 
similar rate, after student and school characteristics were controlled for. 


e Algebra II completion rates in grade 11 did not differ between students in 
treatment and control schools with a high percentage of racial/ethnic minority 
students. 

e Algebra II completion rates in grade 11 did not differ between students in treat- 
ment and control schools with a high percentage of economically disadvantaged 
students. However, an exploratory analysis suggests that the effects may differ 
between students in low-income schools and students in schools that were not 
low-income schools. 


While this study provides useful information on the role of informing parents about the 
importance of Algebra II, there are a few limitations. First, the study team was able to 
follow students only through grade 11 rather than through the end of high school; however, 
most students who complete Algebra II in Texas are expected to do so before grade 12. 
Second, the control group did not carry on in a business-as-usual manner: parents of stu- 
dents in control schools received a brochure describing changes to the Texas graduation 
requirements in general and pointing to websites where additional information could be 
found. Third, it was not possible to determine what additional information, if any, parents 
of students in treatment or control schools may have received regarding the importance of 
Algebra II. Having that information might have enabled the study team to hypothesize as 
to why impacts were or were not found. Finally, although it was possible to determine with 
some certainty that the informational brochures were delivered, the study team could not 
be certain that parents opened the envelopes, read the brochure, shared the brochure with 
their children, or accessed material containing the information contained in the treat- 
ment brochure. 
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Why this study? 


Prior to June 2013, Texas students could choose between two graduation plans, the default 
Recommended High School Plan and the Distinguished High School Plan, with special 
provisions given for students to drop down to the Minimum High School Plan.’ Begin- 
ning with the incoming grade 9 cohort of 2007/08, Texas introduced the 4x4 curriculum 
plan through which students following the recommended and distinguished plans were 
required to take four courses each in English, math (including Algebra II), science, and 
social studies—earning the credits they needed to be admitted to most state universities 
and colleges. 


With the enactment of House Bill 5 (HB 5) in June 2013, these plans were replaced with 
the Foundation High School Program, which encompasses three graduation options (see 
box 1 for definitions of key terms used in the report): 

e The basic Foundation High School Program, which students need permission to 
select and which requires students to complete 22 credits, including three courses 
each in science, social studies, and math. 

e The default Foundation High School Plan plus an endorsement (Foundation High 
School Program Plus Endorsement), which requires students to complete 26 credits 
to graduate, including four courses each in English, math, and science. Algebra II 
is required only for students who earn the science, technology, engineering, and 
math endorsement.* 

e A distinguished level of achievement, which adds Algebra II to the requirements 
of the Foundation High School Program Plus Endorsement, regardless of which 


endorsement a student pursues.° 


The Foundation High School Program was crafted to give students who want to go directly 
into career and technical fields the flexibility to take more classes focused on their interests 
instead of college preparation classes. 


HB 5 also changed the automatic admission policy (known as the Top 10 Percent Rule) of 
Texas public colleges and universities. The policy now offers admission to all Texas high 
school students who graduate in the top 10 percent of their class and who earn a distin- 
guished level of achievement, starting with the 2014/15 incoming cohort of grade 9 stu- 
dents.° Most public universities in Texas still require students to have completed Algebra II 
in order to gain admission, despite the new graduation plans. 


Research suggests that the children of racial/ethnic minority and low-income parents are 
most likely to be affected by changes in graduation requirements because they are the 
most likely to complete only the minimum graduation requirements (Chaney et al., 1997; 
Domina & Saldana, 2011; Saw & Broda, 2012; Schiller & Muller, 2003). As such, remov- 
ing the Algebra II graduation requirement for all high school students may disproportion- 
ately affect racial/ethnic minority and economically disadvantaged students’ eligibility for 
admission to public four-year colleges and universities in Texas. (See appendix A for a 
literature review on the influence of graduation requirements on student course taking, 
relationships between Algebra I] completion and college outcomes, relationships between 
Algebra I] completion and career outcomes, and the role of information dissemination on 
student choice.) 


Without 
information on the 
alignment between 
the new high 
school graduation 
requirements and 
college entrance 
requirements, 
parents— 
particularly racial/ 
ethnic minority 
and low-income 
parents—may not 
be able to help 
their children make 
informed choices 


Box 1. Key terms 


Algebra II completion. Refers to students who were enrolled in Algebra Il for one year by the 
end of grade 11, regardless of whether they passed the course. 


Automatic admission policy/Top 10 Percent Rule. Policy whereby students who graduate in the 
top 10 percent of their class and who earn a distinguished level of achievement are eligible for 
automatic admission to Texas public colleges and universities. 


Distinguished level of achievement. An option of the Foundation High School Program that can 
be earned by completing Algebra II as well as the requirements of the Foundation High School 
Program Plus Endorsement. 


Economically disadvantaged. A student who is eligible for the federal school lunch program. 


Foundation High School Program. The minimum 22-credit curriculum plan for Texas high 
school graduation, which requires four credits in English; three credits each in math, science, 
and social studies; two credits in a language other than English; a credit each in fine arts and 
physical education; and five electives. 


Foundation High School Program Plus Endorsement. The default 26-credit curriculum plan for 
Texas high school graduation, which requires four credits each in math, English, and science; 
three credits in social studies; two credits in a language other than English; a credit each 
in fine arts and physical education; and seven electives focused around a selected career 
endorsement. 


Low-income school. A school in which the percentage of economically disadvantaged students 
is in the top third of all schools in the state. 


High-racial/ethnic minority school. A school in which the percentage of racial/ethnic minority 
students is in the top third of all schools in the state. 


Without information on the alignment between the new high school graduation require- 
ments and college entrance requirements, parents—particularly racial/ethnic minority and 
low-income parents—may not be able to help their children make well informed choices. 


Because Algebra II was no longer required for high school graduation but was still required 
for automatic college admission under the Top 10 Percent Rule, the chair of the State 
Board of Education was concerned that students and parents would miss critical infor 
mation in planning for high school and college. The change in state graduation require- 
ments provided an opportunity to generate rigorous evidence about the role of information 
dissemination in students’ course completion. Information from this study provides the 
State Board of Education, the Texas Education Agency, and the Texas Higher Education 
Coordinating Board with information regarding their attempts to reduce the number of 
students who do not complete Algebra II. 


What the study examined 


Two research questions guided this study. 

The first was confirmatory: 

1. Does providing parents and guardians with information about the role of Algebra II 
in college admission have an impact on the percentage of students who complete 
Algebra II in grade 11? 


The second was exploratory: 


2. Do the impacts vary by the percentage of students in a school who are racial/ethnic 
minorities or economically disadvantaged? 


The data and methods used to answer the research questions are summarized in box 2 and 
detailed in appendix B. 


Box 2. Data, sampling, and methods 


Data 

The study used longitudinal student- and school-level datasets provided by the Texas Education 
Agency (TEA). Specifically, the study used data from the TEA Public Education Information Man- 
agement System, statewide assessment files, and Texas Academic Performance Report files. 
The Public Education Information Management System contains student-level data on student 
enrollment and demographic characteristics, special program participation, and course com- 
pletion. The Texas Academic Performance Report files contain organizational data for schools 
and districts, such as percentage of students in each racial/ethnic group, percentage of eco- 
nomically disadvantaged students, percentage of students in special education, percentage 
of English learner students, and percentage of students who passed state achievement tests. 
The statewide assessment files contain student-level data on the State of Texas Assessments 
of Academic Readiness™, the Texas state achievement test. 

The study also used information from a website review performed as part of a Regional 
Educational Laboratory Southwest technical assistance project conducted on behalf of TEA in 
fall 2014. In that project researchers reviewed the websites of all 1,026 public school districts 
in Texas to assess whether districts required or encouraged students to complete Algebra Il. 
(See appendix B for additional information.) 


Sampling 

School districts with at least one high school were recruited from across Texas to participate 
in the study. The recruiting strategy for the study followed two objectives: to recruit at least two 
high schools in each of the 20 Education Service Center regions across the state (see http:// 
tea.texas.gov/regional_services/esc/ for a map of regions)* and to recruit high schools that 
did not require or strongly encourage students to complete Algebra II—high schools in which 
it was believed that the study informational materials could make the largest impact. Schools 
were informed that the study was being conducted as a randomized controlled trial in which half 
of participating schools would be randomly assigned to a treatment group and the other half 


(continued) 


Box 2. Data, sampling, and methods (continued) 


would be assigned to the control group. After random assignment, schools in the treatment 
group were provided informational brochures about the role of Algebra II in college admission to 
send to parents and guardians. Schools in the control group were provided with informational 
brochures describing the changes to the high school graduation requirements, including the 
elimination of Algebra II as a graduation requirement, to send to parents and guardians. 

Recruitment was conducted though research applications to the district, email and tele- 
phone contact, and public information requests.2 Recruitment was conducted at the high 
school or district level depending on the requirements of the district. A total of 116 high 
schools in 63 districts were recruited to participate and were randomly assigned to the treat- 
ment and control groups within each Education Service Center region. (When more than one 
school from a district was recruited, blocking for randomization was done by district and Edu- 
cation Service Center region.) Of those 116 schools, 109—54 treatment schools and 55 
control schools—participated in the study. Compared with all other high schools across Texas, 
the participating schools had higher percentages of racial/ethnic minority students, econom- 
ically disadvantaged students, English learner students, and lower achieving students. Study 
schools were also substantially larger, on average. 

To track whether schools mailed the brochures to parents and guardians, each participat- 
ing school was provided with an envelope addressed to the study team. This envelope was 
included in the box of envelopes to be disseminated to parents and guardians. The study team 
asked participating schools to mail these envelopes at the same time they mailed the other 
envelopes, which had the study team address as the return address so that if the parent/ 
guardian address was not correct, they would be returned to the study team. Any envelopes 
received by the study team were used to determine whether schools mailed the materials to 
the parents and guardians and when the materials had been mailed. The study team placed 
telephone calls to schools for which no envelope was received by the end of March. School 
staff responding to the telephone calls verified whether the envelopes to parents and guard- 
ians had been mailed and the mailing date. 

Students were included in the analyses if they were enrolled in grade 10 in a study high 
school during the 2015/16 school year, were enrolled in a Texas public high school during the 
2016/17 school year, and had not completed Algebra II before the 2016/17 school year.? A 
total of 29,483 students were included in the analyses—14,415 in treatment schools and 
15,068 in control schools. 


Methods 

To answer the confirmatory research question, a multilevel regression model was used to 
compare Algebra II completion rates by the end of grade 11 for students in treatment schools 
and students in control schools. To answer the exploratory research question, interaction 
terms were included in the model to identify differential impacts for high—racial/ethnic minority 
and low-income schools. The models employ random effects to control for clustering by school. 
Results are reported in figures 1-3 in the main text as the probability of completing Algebra II 
in grade 11; log odds ratios, odds ratios, and standard errors are shown in appendix E. 


Notes 

1. Despite this strategy, only one high school was recruited in region 1 (Edinburg). 

2. Texas districts are required to respond to information requests from the public. The study team obtained 
high school directory information for grade 10 students after submitting public information requests to high 
school districts in Texas. 

3. The percentage of students who completed Algebra II prior to the 2015/16 school was very similar in treat- 
ment schools (26 percent) and control schools (25 percent). 
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The interventions for the study consisted of informational brochures for parents and guard- 
ians of grade 10 students. The brochures were designed with the assistance of a marketing 
professor at the University of Pittsburgh to have a professional appearance and minimal 
text. They contained links to websites where parents could find more information about 
changes to the high school graduation requirements. 


The brochure sent to parents and guardians of students in treatment schools was a two- 
page (one double-sided sheet of paper), trifold full-color brochure displaying information 
about the role of Algebra II in four-year college admission (see appendix C). It outlined the 
course requirements for each of the new high school graduation plans and how they related 
to the admission requirements of public four-year colleges and universities in Texas. It also 
highlighted the mismatch between graduation requirements and college and university 
admission requirements and stated that, beginning with the 2014/15 cohort of grade 9 stu- 
dents, only students who complete the distinguished level of achievement are eligible for 
the state’s automatic admission policy. English and Spanish versions of the brochure were 
produced, and all parents of students in treatment schools received both versions. 


The brochure that was sent to parents and guardians of students in control schools was a 
one-page, full-color brochure briefly outlining the primary changes to the high school grad- 
uation requirements, including that only the science, technology, engineering, and math 
endorsement and the distinguished level of achievement require completion of Algebra II 
(see appendix D). The information in the brochure was also available on the Texas Edu- 
cation Agency website. English and Spanish versions of the brochure were produced (with 
one language printed on each side), and all parents of students in control schools received 
both versions. 


The brochures were sent through the U.S. mail, and all envelopes containing the bro- 
chures were sealed, stamped, and printed with the Texas Education Agency logo to encour 
age parents to open them. Dissemination of the brochures coincided with the time period 
when grade 10 students were selecting courses for their junior year: January—March 2016. 
(See appendix B for additional details about dissemination of the brochures.) 


What the study found 


This section presents the main findings of the study. 


Informing parents about the role of Algebra Il in college admission had no clear impact on Algebra II 
completion rates for grade 11 students 


After student background characteristics and school characteristics were controlled for, 
students in treatment and control schools completed Algebra II in grade 11 at a similar 
rate. The adjusted predicted probability of completing Algebra II in grade 11 was .80 for 
students in treatment schools and .78 for students in control schools (figure 1). Although 
the Algebra II completion rate for treatment group students was higher than that of stu- 
dents in control schools, the difference was not statistically significant. (See table El in 
appendix E for log odds ratios, odds ratios, and standard errors for all variables in the 


model.) 


The adjusted 
predicted 
probability of 
completing 
Algebra II in 
grade 11 was 
.80 for students 
in treatment 
schools and .78 
for students in 
control schools 


Figure 1. The adjusted predicted probability of completing Algebra Il in grade 11 
in Texas was not statistically significantly different for students in treatment and 
control schools, 2016/17 
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Note: See appendix E for log odds ratios, odds ratios, and standard errors for all variables in the model. 


Source: Authors’ calculations based on data from the Texas Education Agency Public Education Information 
Management System, statewide assessment files, and Texas Academic Performance Report files. 


In schools with a high percentage of racial/ethnic minority students, informing parents about the 
role of Algebra Il in college admission had no clear impact on Algebra II completion rates for grade 
11 students 


After school and student characteristics were controlled for, informing parents about the 
role of Algebra II in college admission did not have a statistically significant impact on 
students in high—racial/ethnic minority schools or students in schools that were not high— 
racial/ethnic minority schools. The analyses also did not find differences in impacts based 
on the percentage of racial/ethnic minority students in the school. In high—racial/ethnic 
minority schools the adjusted predicted probability of completing Algebra II in grade 11 
was .83 for students in control schools and .82 for students in treatment schools, after 
student and school characteristics were controlled for. In schools that were not high—racial/ 
ethnic minority schools the adjusted predicted probability was .78 for students in control 
schools and .82 for students in treatment schools, after student and school characteristics 
were controlled for (figure 2). The differences between treatment and control groups in 
each type of school were not statistically significant. (See table E2 in appendix E for log 
odds ratios, odds ratios, and standard errors for all variables in the model.) 


Algebra II completion rates in grade 11 did not differ between students in low-income treatment 
and control schools, but an exploratory analysis suggests that the effects differ between students in 
low-income schools and students in schools that were not low-income schools 


The impacts of the intervention were not statistically significant for students in either 
low-income schools or schools that were not low-income schools. However, the statistical- 
ly significant interaction between treatment status and school low-income status suggests 
that the intervention’s effect on students in low-income schools differed from its effect on 


In high-racial/ 
ethnic minority 
schools the 
adjusted predicted 
probability of 
completing 
Algebra II in 

grade 11 was .83 
for students in 
control schools and 
-82 for students 

in treatment 
schools; in schools 
that were not 
high-racial/ 
ethnic minority 
schools the 
adjusted predicted 
probability was 
.78 for students in 
control schools and 
.82 for students in 
treatment schools 


Figure 2. For both high-racial/ethnic minority schools and schools that were not high-racial/ethnic 
minority schools in Texas, the adjusted predicted probability of completing Algebra Il in grade 11 was 
not statistically significantly different for students in treatment and control schools, 2016/17 


Adjusted predicted probability of completing Algebra II in grade 11 
1.0 - 


Treatment schools Control schools Treatment schools Control schools 
High-racial/ethnic minority Not high-racial/ethnic minority 
Note: High-racial/ethnic minority schools are schools in which the percentage of racial/ethnic minority students is in the top third of 
the state. See appendix E for log odds ratios, odds ratios, and standard errors for all variables in the model. 


Source: Authors’ calculations based data from the Texas Education Agency Public Education Information Management System, 
statewide assessment files, and Texas Academic Performance Report files. 


Figure 3. For both low-income schools and schools that were not low-income schools in Texas, the 
adjusted predicted probability of completing Algebra Il in grade 11 was not statistically significantly 
different for students in treatment and control schools, but the intervention’s effect may have differed 
for students in low-income schools and students in schools that were not low-income schools, 2016/17 


Adjusted predicted probability of completing Algebra II in grade 11 
1.0 - 


Treatment schools Control schools Treatment schools Control schools 
Low-income Not low-income 
Note: Low-income schools are schools in which the percentage of economically disadvantaged students is in the top third of all schools 


in the state. The interaction between treatment status and school low-income status was significant at p < .05. See table E3 in 
appendix E for log odds ratios, odds ratios, and standard errors for all variables in the model. 


Source: Authors’ calculations based on data from the Texas Education Agency Public Education Information Management System, 
statewide assessment files, and Texas Academic Performance Report files. 


students in schools that were not low-income schools. Specifically, in low-income schools 
the Algebra II completion rate in grade 11 was lower for students in treatment schools 
than for students in control schools, but in schools that were not low-income schools, 
rates were higher for students in treatment schools than for students in control schools. 
The probability of completing Algebra II in grade 11 was .71 for students in low-income 
treatment schools and .79 for students in low-income control schools; the probability was 
.85 for students in treatment schools that were not low-income schools and .77 for students 
in control schools that were not low-income schools (figure 3). (See table E3 in appendix E 
for log odds ratios, odds ratios, and standard errors for all variables in the model.) 


Implications of the study findings 


Overall, the study found no statistically significant differences in Algebra II completion 
rates in grade 11 between students in treatment and control schools, nor did it find treat- 
ment effects when high-racial/ethnic minority schools and schools that were not high— 
racial/ethnic minority schools were analyzed separately or when low-income schools and 
schools that were not low-income schools were analyzed separately. 


The results are not surprising in light of the findings of a recent Regional Educational Lab- 
oratory Southwest descriptive study on statewide trends in Algebra II completion by grade 
11 before and after changes to Texas state graduation requirements (Stoker, Mellor, & Sul 
livan, 2018). That study found that the statewide rate of Algebra II completion by grade 11 
did not change following the introduction of the Foundation High School Program and 
concluded: “While HB 5 made room for flexibility in the high school course math cur 
riculum, many districts continued to place students in Algebra I, a math course that is a 
prerequisite for admission to most colleges and universities in Texas” (p. 10). 


However, an analysis conducted to answer the current study’s exploratory research question 
found a statistically significant interaction between schoollevel treatment and low-income 
school status. In particular, while the estimated impacts of the treatment were not statis- 
tically significant for students in low-income schools or for students in schools that were 
not low-income schools, the interaction suggested a less positive impact for students in low- 
income schools. Interpreting the statistically significant interaction is difficult. However, 
the results seem to suggest that the informational materials functioned differently depend- 
ing on the type of school that distributed them—low-income or not low-income. Addi- 
tional research could help parse this out. Knowing that parents and guardians of students 
in schools with and in schools without a high percentage of economically disadvantaged 
students respond differently to the two types of brochures could help the Texas Education 
Agency better design and target informational materials for parents and guardians. 


Limitations of the study 


This study has four main limitations. 


First, the study team was able to follow students only through grade 11. While most students 
in Texas—over 75 percent—complete Algebra II by the end of grade 11 (Stoker et al., 2018), 
some complete it in grade 12. Prior to HB 5, about 5 percent of students completed Algebra II 
in grade 12. Given the timeline of this study, it is not possible to determine whether the bro- 
chures have an impact on students who may complete Algebra II in grade 12. 
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The statistically 
significant 
interaction 
between school- 
level treatment and 
low-income school 
status seems to 
suggest that the 
informational 
materials 
functioned 
differently 
depending on the 
type of school they 
were distributed 
to—low-income or 
not low-income 


Second, the control group did not carry on in a business-as-usual manner. Rather, parents 
of students in control schools received a brochure describing changes to the Texas high 
school graduation requirements, including changes to the math requirements (that 
Algebra II is required only for students completing the science, technology, engineering, 
and math endorsement or the distinguished level of achievement). It is possible that receipt 
of the brochure prompted parents to look into changes in the graduation requirements, 
including the repercussions of not completing Algebra II, more than they would have if 
they had not received the brochure. It is also possible that receipt of the brochure led 
parents of students in control schools to believe that taking Algebra II is not necessary for 
their student. 


Third, although it was possible to determine with some certainty that the informational 
brochures were delivered, it was not possible to determine whether parents opened the 
envelopes, read the content of the brochure, or shared the brochure with their children. It 
was beyond the scope of the study to contact parents to determine whether they accessed 
the information in the envelopes. It was also beyond the scope of the study to determine 
whether parents accessed other information about the importance of Algebra II. This is 
the case with most interventions of this type. 


Fourth, although the study team identified schools to recruit for the study based on infor- 
mation provided on district websites—whether the district was promoting or requiring 
completion of Algebra II—the website review was conducted several months prior to 
recruitment. It is possible that the information on the websites did not reflect the district 
policy and practices in place at the time that the students in the study cohort were choos- 
ing courses for grade 11. It is also possible that districts had been promoting Algebra II 
through means other than the district website. This may have reduced the impact that 
the brochure had on students in treatment schools, especially if more control schools were 
pushing students toward Algebra II. 


The control group 
did not carry on 

in a business-as- 
usual manner; 
rather, parents of 
students in control 
schools received 

a brochure 
describing changes 
to the Texas high 
school graduation 
requirements, 
including changes 
to the math 
requirements 


Appendix A. Literature review 


This appendix describes prior research on the influence of graduation requirements on 
student course taking, relationships between Algebra II completion and college outcomes, 
relationships between Algebra II completion and career outcomes, and the role of informa- 
tion dissemination on school choice. 


Influence of graduation requirements on student course taking 


Graduation requirements have a strong influence on the courses that students take and 
the courses that schools offer. The publication of A Nation at Risk (National Commission 
on Excellence in Education, 1983) had a profound effect on perceptions of the types of 
knowledge and skills that students need to be prepared for college or career. As a result, 
states began to change the number of core content courses that students needed to com- 
plete to graduate from high school. Within 10 years of publication of A Nation at Risk, 45 
states raised high school graduation requirements, with the most dramatic increases in 
math and science (Stevenson & Schiller, 1999). Considerable research in the late 1990s 
and early 2000s investigated the influence of these higher graduation requirements on stu- 
dents’ math course taking and achievement. Overall, these studies found that high school 
graduation requirements influenced the level or number of math courses that students 
completed (Clune & White, 1992; Finn, Gerber, & Wang, 2002; Schiller & Muller, 2003; 
Teitelbaum, 2003); however, studies were mixed as to whether graduation requirements 
influenced student achievement, as measured by grades and test scores. Results of most 
studies suggested that although increases in math level were associated with gains in math 
achievement, increases in the number of math courses students take were not (Chaney 
et al., 1997; Teitelbaum, 2003). 


Several studies have found that graduation requirements have a disproportionate influence 
on marginal students (those who completed only the minimum graduation requirements; 
Chaney et al., 1997) and racial/ethnic minority and economically disadvantaged students 
(Domina & Saldana, 2011; Saw & Broda, 2012; Schiller & Muller, 2003). Domina and 
Saldana (2011) found that the national trend toward academic intensification dispropor- 
tionately influenced the math course completion patterns of Black, Hispanic, and low-so- 
cioeconomic status students. Specifically, they found that due to increasing standards, 
between 1982 and 2004 the enrollment rate in advanced math courses grew faster for 
Black and Hispanic students than for White students. With regard to Algebra II comple- 
tion, even after differences in family background and grade 10 test scores were controlled 
for, the odds of completing Algebra II grew faster for Black and Hispanic students than 
for White students. The authors found similar evidence to suggest that curricular intensi- 
fication also boosted economically disadvantaged and low-achieving students’ Algebra II 
completion rates. Between 1982 and 2004 the gap in Algebra II completion rates between 
economically disadvantaged students and non—economically disadvantaged students nar- 
rowed from 16 percentage points to 11, and the Algebra II completion rate for low-achiev- 
ing students surged nearly 30 percentage points. 


These increases in math course taking coincided with rising college enrollment rates 
for Black and Hispanic students. Between 1983 and 2011 the percentage of recent high 
school graduates who enrolled in a two- or four-year college increased from 38 percent to 
67 percent for Black students and from 54 percent to 67 percent for Hispanic students. 
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These patterns suggest that increases in high school graduation requirements have had 
an influence on college preparation and enrollment for these students (Snyder & Dillow, 
2012). 


Relationships between Algebra Il completion and college outcomes 


Research on the relationships between high school math completion and college success 
has found positive, statistically significant relationships between completion of higher level 
math courses and college enrollment, persistence, and completion. For example, Horn and 
Nunez (2000) found that students who completed Algebra II or higher were significantly 
more likely to enroll in a four-year college within two years of high school graduation than 
were students who did not complete Algebra II or higher. These effects were particularly 
strong for students whose parents completed no more than a high school education. Simi- 
larly, Adelman (1999) found that of all precollege curricula, the highest level of math that 
students study in high school had the strongest influence on bachelor’s degree completion 
—finishing a course beyond the level of Algebra II (for example trigonometry or pre- 
calculus) more than doubled the odds that a student who entered postsecondary education 
would complete a bachelor’s degree. Adelman’s (2006) follow-up work found that every step 
up the math course ladder multiplied the odds of earning a bachelor’s degree by roughly 
2.5. He again found that although students who completed Algebra II or higher were more 
likely to earn a bachelor’s degree than were students who completed geometry or lower 
math courses, students who moved one step above Algebra II to trigonometry were consid- 
erably more likely to earn a bachelor’s degree. Horn, Kojaku, and Carroll (2001) also found 
a consistent advantage for students who completed a rigorous high school curriculum, 
including higher level math courses. These students were more likely to remain in their 
initial higher education institution and stay on track for a bachelor’s degree. Moreover, 
students who completed a rigorous curriculum in high school were statistically significantly 
less likely to report taking remedial coursework in college. 


Relationships between Algebra II completion and career outcomes 


Gaertner, Kim, DesJardins, and McClarty (2013) used data from the National Education 
Longitudinal Study (NELS) of 1988 and the Education Longitudinal Study (ELS) of 2002 
to investigate relationships between high school math course taking and college and career 
outcomes two years after high school graduation. The authors found that Algebra I] com- 
pletion was not associated with any of the career outcomes using either NELS or ELS data, 
with one exception. With the NELS data, Algebra II completion was a statistically sig- 
nificant and positive predictor of annual earnings change. However, the analyses did find 
statistically significant relationships between Algebra II completion and several college 
outcomes. Using the NELS dataset, the authors found statistically significant, positive rela- 
tionships between Algebra II completion and admission to college, cumulative grade point 
average, second-year retention, and college graduation. With the ELS data, Algebra II was 
statistically significant and positively related to admission to a highly selective college.’ 
Although the authors concluded that Algebra II completion appears to be more important 
for college outcomes than for career outcomes, it is difficult to tell given the short time 
period in the study to examine career outcomes. Most students will not settle into a career 
immediately out of high school. 
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The role of information dissemination on student choice 


Tailoring or targeting information to individuals has been shown to be successful at influ- 
encing choice in a wide variety of fields (Dougherty et al., 2010; Kreuter, Stretcher, & Glass- 
man, 1999; Resnicow et al., 2009). Studies from fields outside of education, such as health 
sciences and political science, suggest that simplified information presentation, including 
pictorial representations, basic wording, and larger and clearer print, can improve infor 
mation accessibility and use (Davidhizar & Brownson, 2000; Hastings, Van Weelden, & 
Weinstein, 2007; Hastings & Weinstein, 2008; Koning & Van der Wiel, 2010). Moreover, 
studies on information readability in health care have shown that the way information is 
presented and disseminated can significantly influence the choices people make (Hibbard 
& Peters, 2003; Holmes-Rovner, Llewellyn-Thomas, Coulter, O’?Connor, & Rovener, 2001; 
Weener & Girasek, 2003). 


Three large-scale experimental studies have looked at the impact of providing parents 
with school information on subsequent outcomes. Two of the experimental studies looked 
at links between school information and parent choice. Both studies found that provid- 
ing simplified information directly to a select group of parents increases the probability of 
those parents choosing higher achieving schools for their children (Hastings et al., 2007; 
Hastings & Weinstein, 2008). One experimental study examined the role of providing 
parents with information about the importance of math and science in daily life and for 
various careers, as well as information about how parents could communicate with their 
children and personalize the relevance of math and science for them (Harackiewicz, Rozek, 
Hulleman, & Hyde, 2012). The authors found that students in the experimental group 
whose parents had received the information took statistically significantly more math and 
science courses during their last two years of high school. They also found that students 
who received the brochures were statistically significantly more likely to take addition- 
al elective, advanced math and science courses (that is Algebra II, precalculus, calculus, 
chemistry, and physics). 
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Appendix B. Data and methodology 


This appendix describes the data sources, sample, dissemination of materials, and analysis. 
Data 


The study used longitudinal student- and school-level datasets provided by the Texas Edu- 
cation Agency (TEA). Specifically, the study used data from the TEA Public Education 
Information Management System, statewide assessment files, and Texas Academic Perfor 
mance Report files for the 2014/15—2016/17 school years. 


The study also used information from a website review performed as part of a Regional 
Educational Laboratory (REL) Southwest technical assistance project conducted on behalf 
of TEA in fall 2014. In that project, researchers reviewed the websites of all 1,026 public 
school districts in Texas to assess whether districts required or encouraged students to 
complete Algebra II (box B1). 


Box B1. Website review 


From July to October 2014 Regional Educational Laboratory Southwest researchers conducted 

Internet searches for each of the 1,026 public school districts in Texas. Researchers reviewed 

each district website for information regarding Texas House Bill 5 (HB 5), particularly as it per- 

tained to Algebra Il and high school graduation requirements. On each website, researchers: 

¢ Conducted keyword searches using the terms “high school graduation requirements,” 
“high school graduation plans,” “HB 5 graduation plans,” “HB 5 endorsements,” “founda- 
tion high school plan,” “FHSP,” and “endorsement” within the district search engine. 

e Reviewed the district’s board meeting notes from Spring 2014.1 

e Reviewed the student and parent sections of the website. 


Depending on the information found, researchers recorded: 

e The default diploma option (that is, Foundation High School Program, Foundation High 
School Program Plus Endorsement, or distinguished level of achievement), if any, assigned 
to grade 9 students entering district high schools in the 2014/15 school year. 

¢ The endorsement options being offered in the district (that is, arts and humanities; 
business and industry; public services; science, technology, engineering, and math; or 
multidisciplinary). 

¢ How information about the new graduation requirements was being disseminated, if at all 
(for example, by website, email, fliers, or parent meetings). 


Approximately 11 percent of districts encouraged students to complete a distinguished level 
of achievement, for which students must complete Algebra II, or made that the default option. 

These data were used to identify schools for recruitment in the current study, though 
some of the information may have changed between 2014, when it was collected, and spring 
2016, when the students in the current study chose their grade 11 courses. 


Notes 


1. Several school districts provided information in board minutes about how the districts are responding to HB 
5 with regard to diploma plan placement and endorsements offered. 
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Sample 


The sampling strategy for the study followed two objectives: to recruit at least two high 
schools from each of the 20 Education Service Center regions across Texas and to recruit 
high schools that, based on the website review discussed in box Bl, did not require or 
strongly encourage students to complete Algebra II—high schools in which it was believed 
that the study materials could make the largest impact. Schools that strongly encouraged 
students to complete Algebra II and schools listed as an Early College High School or a 
magnet school were omitted. In addition, high schools that did not cover grades 9-12 were 
omitted; however, high schools with any grade configuration that included all of these 
grades were recruited. 


Recruitment for the study was conducted at the high school and district levels. The study 
team used three techniques to recruit districts and high schools. 

e Research requests. As required by district policy, the study team submitted research 
requests to five very large districts in winter 2015 and received district approval 
from three of them. One district provided Excel spreadsheets containing student 
names and addresses for all high schools in the district, and the other two districts 
required the study team to contact individual schools to recruit them to partici- 
pate in the study. A total of 20 schools from three districts were recruited using 
research requests. 

e Email messages and telephone calls. For districts not requiring a research request, 
the study team used email and telephone calls to recruit high schools. Starting 
in fall 2015, the study team began working with the Texas Association of School 
Administrators (TASA) to recruit high schools to participate in the study. TASA’s 
membership includes all district superintendents in Texas. One of TASA’s goals is 
to ensure that districts are getting as much information as possible when legislative 
changes affect students. During the first week of October, TASA introduced the 
study to district superintendents throughout the state in its online newsletter. The 
newsletter contained a short description of the study and invited superintendents 
to share this information with principals in their districts. A week later, the study 
team sent a follow-up email to all superintendents in the state providing informa- 
tion about the study and inviting them to participate. Superintendents were asked 
to contact the study team if they were interested in having their high schools 
participate in the study or to forward the email to high school principals in their 
districts. About one week later, similar emails were distributed to high school prin- 
cipals throughout the state to introduce the study and invite them to participate. 

Both sets of emails outlined the goals of the study, the content of the informa- 
tional materials, and schools’ roles in information dissemination. Principals were 
made aware that the study was being conducted as a randomized controlled trial 
and that half of participating schools would receive the informational materials 
focused on the role of Algebra II in college admission, while the other half would 
receive informational materials describing general changes to the graduation 
requirements. Principals were informed that all participating schools would have 
access to all study materials at the conclusion of the study—the 2017/18 school 
year. Interested principals and superintendents were directed to contact the study 
team for additional information. 

Three weeks after the initial emails were distributed to superintendents and 
principals, the study team conducted telephone calls to recruit additional high 
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schools in Education Service Center regions from which two schools had not 
already been successfully recruited. A total of 51 high schools were recruited using 
the email and telephone campaign. 

e Public information requests. While conducting school recruitment, the study 
team learned that several districts could not consent to participate in the study 
because the study did not meet the district definition of research—because it did 
not include a district-level data collection component. As such, the study team 
was directed to submit public information requests to obtain spreadsheets con- 
taining publicly available student names and addresses for students in district 
high schools.® The directories are regularly used by businesses, colleges, and other 
entities to contact students and parents in area schools. Using this process, the 
study team obtained data for 45 public high schools and mailed the materials to 
students directly. Before the informational materials were mailed, emails were sent 
to all the schools asking them to contact the study team if they did not want the 
materials mailed to their students. Only one school contacted the study team for 
additional information about the study. Upon hearing the details of the study, the 
school agreed to allow the materials to be mailed to students. 


A total of 116 schools in 63 districts were recruited to participate in the study. Two schools 
were recruited in each Education Service Center region except region 1, in which only one 
school was recruited.’ 


After the recruitment period closed, high schools were randomized into treatment and 
control groups. Simple randomization within a region was conducted using participat- 
ing high schools in each Education Service Center region (table Bl). The one school in 
region | was assigned to the treatment condition to even out the number of treatment and 
control schools.!° Thus, both the treatment and control groups contained 58 high schools. 


Of the 116 schools that were recruited, 109 participated in the study. Seven schools ver- 
bally agreed to participate but failed to sign the written consent form and either informed 
the study team that they did not plan to mail the materials or did not respond to telephone 
inquiries. These schools were dropped from the study—four treatment schools and three 
control schools. The final analytic sample contained 54 treatment schools (14,415 students) 
and 55 control schools (15,068 students). Overall attrition for schools was 6.0 percent, with 
differential attrition of 1.7 percent (table B2). Overall attrition for students was 6.5 percent, 
with differential attrition of 3.0 percent. 


All students who began grade 9 during the 2014/15 school year (the first cohort of students 
to be required to graduate under the Foundation High School Program), were in grade 10 
and present in Texas public high schools during the 2015/16 school year, and remained 
enrolled in a Texas public high school during the 2016/17 school year, including students in 
special education and English learner students, were eligible to be included in the study if 
their school opted to participate.!! Students in grade 10 who were not part of the incoming 
grade 9 cohort of 2014/15 in participating schools were excluded from the study, although 
they may have received study materials. Students who completed Algebra II before the 
2016/17 school year were also excluded because the informational brochures could not 
have an impact on students who had already completed Algebra II. Table B3 shows the 
number of students meeting these qualifications. 


Table B1. Number of study schools in each Texas Education Service Center region 
recruited to participate and in the analytic sample, 2015/16 


Schools in the analytic sample 


Education Service 


Center region Schools recruited All schools Treatment schools Control schools 
1 1 1 

2 2 2 1 1 
3 3 3 1 2 
4 14 14 6 8 
5 2 2 1 1 
6 5 4 2 2 
7 3 3 2 1 
8 4 4 2 2 
9 2 2 al it 
10 17 17 8 9 
11 16 14 7 7 
12 8 7 3 4 
13 14 13 7 6 
14 3 3 2 1 
15 2 2 1 1 
16 3 3 1 2 
17 3 3 1 2 
18 6 6 3 3 
19 2 1 1 (0) 
20 6 5 3 2 
Total 116 109 54 50 


Source: Authors’ compilation. 


Table B2. Sample attrition from treatment and control schools in Texas, 2016/17 


Number recruited Number in analytic sample ; F 
ere ae 2 ara eee Se eda Overall Differential 
Treatment Control Treatment Control ladda) attrition? 
group group group group (percent) (percent) 
Schools 58 58 54 55 6.0 A:T 
Students 15,163 16,363 14,415 15,068 6.5 3.0 


a. Absolute values. 


Source: Authors’ compilation. 


Since school recruitment efforts focused on schools that did not require Algebra II or 
include materials on their district websites specifically endorsing Algebra II, the sample 
was not expected to be representative of all high schools in Texas with grades 9-12. The 
study sample contains higher percentages of racial/ethnic minority students, economically 
disadvantaged students, English learner students, and lower achieving students than did 
all other Texas public high schools (table B4). Participating schools were also substantially 
larger, on average, than all other Texas public high schools. 


Following What Works Clearinghouse (WWC) guidelines, baseline equivalence on key 
demographic and achievement variables was assessed for the analytic sample. Table B5 
presents descriptive data on these variables for students in treatment and control schools, 
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Table B3. Number of students and schools in Texas participating in the study, 
2016/17 


Treatment Control 


Statistic Total group group 
Recruitment 

Number of schools 116 58 58 
Number of students who were enrolled in grade 10 in 2015/16 45,586 22,045 23,541 


Number of students who were also enrolled in grade 11 in 2016/17 42,378 20,535 21,843 
Number of students from who did not take Algebra II before 2016/17 31,526 15,163 16,363 
Analytic sample 

Number of schools 109 54 55 
Number of students enrolled in schools that participated in the study 29,483 14,415 15,068 


Source: Authors’ compilation based on data from the Texas Education Agency Public Education Information 
Management System. 


Table B4. Comparison of study schools to all other public high schools in Texas, 
2016/17 


All other Texas 


public high 

Study schools schools 
Characteristic (me Ko}:)) (ee pd: )) Difference 
Average total enrollment 1,468 952 516 
Percentage of students by race/ethnicity 
American Indian 0.30 0.44 -0.14 
Asian 2.17 2.08 0.09 
Black 17:72 8.96 8.76 
Hispanic 49.93 43.45 6.48 
Pacific Islander 0.16 0.10 0.06 
White 28.03 43.23 -15.20 
Two or more races/ethnicities 11.70 11.20 0.50 
Percentage of students by student characteristics 
Economically disadvantaged 59.31 52.97 6.34 
In special education 9.60 9.27 0.33 
English learner 10.52 5.59 4.93 
Percentage of students by achievement 
Passed Algebra | end-of-course assessment 69.41 73.72 -4.31 


Source: Authors’ calculations based on data from Texas Academic Performance Reports. 


as well as effect size differences between these groups. To assess baseline equivalence 
between students in treatment and control schools, treatment and control group frequen- 
cies and means were entered into a WWC study review guide spreadsheet and effect size 
differences were obtained. Most of the effect size differences were in the acceptable range 
—the absolute value of the effect size was smaller than 0.05. The percentages of Hispanic, 
White, economically disadvantaged, and English learner students had effect size differenc- 
es large enough to warrant inclusion in the analytic models based on WWC criteria. 
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Table B5. Baseline equivalence for the analytic sample of students in Texas, 
2016/17 


Students in Students in 
treatment schools control schools 

Characteristic (N = 14,415) (Be BT) 
Gender 
Female 0.49 0.48 0.02 
Male 0.51 0.52 0.03 
Race/ethnicity 
Asian 0.02 0.02 0.00 
Black 0.21 0.19 0.08 
Hispanic 0.54 0.62 0.20 
Other race/ethnicity 0.02 0.02 0.00 
White 0.20 0.15 0.21 
Student characteristics 
Economically disadvantaged 0.61 0.70 0.24 
In special education 0.114 0.114 0.00 
English learner 0.14 0.16 0.10 
Achievement 
Average Algebra | end-of-course scale score 3,720.39 3,704.03 


(601.11) (556.08) 0.03 


a. Based on absolute differences between the group means. Following What Works Clearinghouse procedures, 
a Cox index was used to calculate the effect sizes for dichotomous outcomes. The Cox index assumes that the 
dichotomous outcome is based on an underlying latent variable and calculates the effect size using a scaled 
difference between log odds of the intervention and control groups. See What Works Clearinghouse (2017) for 
more information. 


Note: Values in parentheses are standard deviations. 


Source: Authors’ compilation using data from the Texas Education Agency Public Education Information Man- 
agement System and State of Texas Assessments of Academic Readiness Algebra | end-of-course data. 


Dissemination of informational materials 


Dissemination of the informational brochures coincided with the time period when 
grade 10 students were selecting courses for their junior year: January-March 2016. The 
envelopes containing the brochures were sealed, stamped, and printed with the TEA logo 
to encourage parents to open them. A copy of the informational brochure that was mailed 
to parents of students in the treatment group is in appendix C, and a copy of the informa- 
tional brochure that was mailed to parents of students in control schools is in appendix D. 


Once schools were recruited into the study, the study team provided them with two options 
for distribution of materials. 


The first was for schools to disseminate the materials themselves. Under this option, school 
staff were asked to provide the study team with the number of grade 10 students for the 
2015/16 school year. The study team then counted out that number of postage-paid enve- 
lopes, plus a few extra, and sent them express mail to the high school using the contact 
name provided by the school. Schools were then asked to print mailing labels for grade 
10 students, attach them to the envelopes, and have the U.S. Postal Service pick them up 
during its daily run. The envelopes were addressed to the “Parents or Guardians of [student 
name]”. Schools were provided with $150 to cover the cost of address labels, printer ink, 
and staff time after the study team verified that the envelopes had been mailed. 
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The boxes of materials also contained an envelope addressed to the study team. Schools 
were asked to mail that envelope along with the student envelopes. Receipt of the letters 
and the postmarks for these letters served as indicators to the study team on whether 
and when schools had mailed the materials. The study team placed telephone calls to 11 
schools that did not mail back the envelopes by the end of March. School staff were asked 
to verify whether the envelopes had been mailed to parents and guardians and the mailing 
date. Additionally, the return address on all envelopes mailed to parents and guardians was 
the REL Southwest office. The study team used the zip codes on the envelopes that were 
returned to REL Southwest as undeliverable to identify which school the envelope was 
associated with. This information was used to verify which schools had mailed the enve- 
lopes to parents and guardians. At least one undeliverable mail envelope was received from 
each participating school. The study team was thus confident that all participating schools 
had mailed the envelopes to parents and guardians. 


The second option was for the study team to disseminate the materials. Under this option, 
schools were asked to provide spreadsheets containing the names and addresses of grade 
10 students. The spreadsheets were then used to print mailing labels, which were affixed 
to the envelopes by the study team. This was done for the schools recruited through public 
information requests, as well as the schools in the district that opted to send the study 
team an Excel file containing students’ names and addresses. 


Schools were never informed of their treatment status. Additionally, it is unlikely that 
a school could have determined its treatment status based on the materials it received 
had someone from the school opened one of the sealed envelopes. The treatment and 
control brochures were not shared with schools before dissemination of the informational 
materials. 


Analysis 


Confirmatory and exploratory analyses were used to assess the impact of the intervention 
on student outcomes. The confirmatory analyses were designed to investigate differences 
in Algebra II completion rates between students in treatment and control schools. The 
exploratory analyses were designed to investigate whether there are differential impacts of 
the treatment for the specified subgroups—high-racial/ethnic minority schools and low- 
income schools. 


Confirmatory analysis. To assess the effect of providing parents with information about 
the role of Algebra II in college admission, a two-level model, with students nested within 
schools, was estimated: 


Student level: 


Log & ( ) gs 
qj 


n, = Boy + B,(Asian,) + B,(Black,,) + B(Hispanic,) + 6, (Other Race,,) + 6.(Special Education,,) 
+ B.(LEP,) + B.(Economic Disadvantage,) + 8,(Algebra I EOC;) 


where LEP indicates that the student is an English learner student, and Algebra I EOC is 
the Algebra I end-of-course assessment. 
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School level: 
Q 


By, = Yo + Yo( Treat.) + Lo 46ESC Region) + Yoox (Cov) + u 


where 9; is the probability that student i in school j completed Algebra II by the end of 
grade 11, Treat, is a binary indicator that takes on the value of 0 for control schools and 1 
for treatment schools, ESC Region, is a set of binary indicator variables that have the value 
of 0 for schools not in the Education Service Center (ESC) region and 1 for schools in the 
ESC region, Cov, are schoollevel covariates (that is, percentage of racial/ethnic minority 
students, percentage of economically disadvantaged students, percentage of students in 
special education, percentage of English learner students, school enrollment, and average 
percentage of students who passed the State of Texas Assessments of Academic Readi- 
ness Algebra I end-of-course exam), and u, are random errors associated with schools. The 
use of random school effects controls for clustering by school. In this model the covariate 
of interest is y,,, which is the treatment effect. A binary indicator for region 1 was not 
included in the model, since only one high school was recruited from that district. In the 
analyses the single high school from region 1 was combined with the single remaining 
school in region 19—one of the region 19 schools dropped out of the study. Region 11 is 
the omitted region, as it had the largest number of schools. All covariates in the model, 
with the exception of the treatment indicator, were grand mean centered. The treatment 
indicator retained the 0/1 coding. 


Exploratory analyses. The subgroup analyses looked at whether the impact of providing 
parents and guardians with information about the role of Algebra II in college admission 
differs by whether the school has a high percentage of racial/ethnic minority students or 
by whether it has a high percentage of low-income students. These analyses build on the 
two-level hierarchical linear model used in the confirmatory analysis: 


Student level: 
2 %; \_ 
E(Y|X) = Los(,$ a Nip 


n, = Boy + B,(Asian,) + B,(Black,,) + 6,f Hispanic, + B,(Other Race,,) + 8.(Special Education, 
+ B.(LEP,,) + B.(Economic Disadvantage,) + 8,.(Algebra I EOC, F) 


School level: 
Q 
Bo = Vogt Yo(Treat,) + ¥o,(High Minority or Low Income) + =" V5 ESC Region) + 


R 
yy o(Cou) a (Treat, * High Minority or Low Income) + 
Ou 


This model includes a term, High Minority, that indicates whether the percentage 
of racial/ethnic minority students in a school is in the top third of the state or a term, 
Low Income, that indicates whether the percentage of economically disadvantaged stu- 
dents is in the top third of the state, depending on the analysis. The model also includes 
Treat, * High Minority or Low Income, in which the treatment indicator is interacted with 
a school-level covariate—either high racial/ethnic minority or low-income, depending on 
the analysis. For example, to investigate differences in the treatment effect by school racial/ 
ethnic composition, a high—racial/ethnic minority school indicator was included, which 
takes on a value of 1 for schools in which the percentage of students in the school who are 
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American Indian, Black, Hispanic, or a race/ethnicity other than Asian or White is in the 
top third of schools in the state, and a value of 0 otherwise. Similar analyses were conduct- 
ed to look at differential impacts for low-income schools. All covariates in these models, 
except the treatment indicators, indicators for high—racial/ethnic minority or low-income 
schools, and interactions between the treatment indicators and indicators for high—racial/ 
ethnic minority or low-income schools, were grand mean centered. The treatment indi- 
cators, indicators for high—racial/ethnic minority or low-income schools, and interactions 
between the treatment indicators and indicators for high—racial/ethnic minority or low- 
income schools retained the 0/1 coding. The coefficient of interest for these analyses is the 
interaction term, Yop,;- 
For the exploratory analyses, separate models were analyzed in which the omitted category 
was reversed in order to estimate the treatment impact for high—racial/ethnic minority 
versus not high-racial/ethnic minority schools and low-income versus not low-income 
schools. That is, in one set of analyses the omitted categories were high—racial/ethnic 
minority schools or low-income schools, while in another set of analyses the omitted cat- 
egories were schools that were not high-racial/ethnic minority schools or not low-income 
schools. The interaction term included matched the analysis. Only the results of the 
analyses in which the omitted categories were schools that were not high—racial/ethnic 
minority schools and schools that were not low-income schools are shown in tables E2 and 
E3 in appendix E. 


The adjusted predicted probabilities shown in figures 2 and 3 in the main text were cal- 
culated using the coefficients obtained from the analyses of the multilevel models. The 
adjusted predicted probabilities for the treatment schools that were not high—racial/ethnic 
minority schools or not low-income schools were calculated by adding the coefficients 
for the intercept and treatment indicator and transforming the log odds ratio to a prob- 
ability. The adjusted predicted probabilities for treatment schools that were high—racial/ 
ethnic minority schools or low-income schools were calculated by adding the coefficients 
for the intercept, treatment indicator, high—racial/ethnic minority or low-income indica- 
tor, and the interaction between treatment and high-racial/ethnic minority or low-income 
and transforming the log odds ratio to a probability. The high—racial/ethnic minority or 
low-income control school adjusted predicted probabilities were calculated by adding the 
coefficients for the intercept and the high—racial/ethnic minority or low-income indicator 
and transforming the log odds ratio to a probability. The adjusted predicted probabilities 
for control schools that were not high-racial/ethnic minority schools or not low-income 
schools were calculated by transforming the log odds ratio for the intercept to a probabil- 
ity. Since all the other variables in the models were grand mean centered, the predicted 
probabilities are based on the mean values of these variables for the overall sample. 
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This appendix includes the informational brochure that was sent to the treatment group. 
The brochure is a trifold. When folded, the panel on the right is the cover of the brochure, 
the panel on the left is folded in, and the panel in the middle is the back of the brochure. 


the high school Class of 2018! 


rele-Leler-iace)a) 
requirements are 
faarclateliare Fa 


Beginning with the graduating class of 2018, 
Texas high school students will have the 
option to complete one of three new 
fele-relUr-va(olaM olt-larsmaat-mmelelavel-1dColameal-laMmdal=s 
Foundation plus Endorsement Plan, or the 
Distinguished Plan. 


With the move to these new graduation 

plans comes new course requirements, 

particularly with regard to mathematics. 

Only students who choose to complete the 

Distinguished Plan or the STEM endorsement 

within the Foundation plus Endorsement Did re) u kn OW. 

Plan will be required to complete Algebra Il. y eee 
For more information: 

Yet, Algebra II is still an admissions 

requirement for most public colleges and Texas Education Agency: 

universities in Texas. http://www.tea.state.tx.us/brochures/ 


Institute for Public School Initiatives 
http://ipsi.utexas.edu/hb5-resources-2/ 


Education Service Center Region 13 
http://www4.esc13.net/cc/house-bill-5 
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Will you have the math to get you to college? 


Not all new graduation plans require Algebra II, but most Texas public universities do! 


Foundation Plan (22 credits) 


English (4 credits) 
Science (3 credits) 
Mathematics (3 credits) 
Social Studies (3 credits) 
Physical Education (1 credit) 
World Language (2 credits) 
Fine Arts (1 credit) 

Electives (5 credits) 


Does NOT require 
Algebra Il 


Will NOT qualify 
for admission to Texas 
public universities 


Foundation plus Endorsement 
(26 credits) 


English (4 credits) 
Science (4 credits) 

Mathematics (4 credits) 

Social Studies (3 credits) 

Physical Education (1 credit) 

World Language or Computer Science | 
Fine Arts (1 credit) ' 
Electives (2 credits) 

Endorsement Requirements. 


Only STEM endorsement 
requires Algebra Il 


Will qualify 
for admission to Texas 
public universities, 


if you complete Algebra Il 


Distinguished (26 credits) 


English (4 credits) 
Science (4 credits) 

Mathematics (4 credits) 

MUST INCLUDE ALGEBRA II 

Social Studies (3 credits) 

Physical Education (1 credit) 

World Language or Computer Science (2 credits) 
Fine Arts (1 credit) 

Electives (2 credits) 

Endorsement Requirements for 1 Endorsement 


Requires Algebra Il 


Will qualify for 
automatic admission 
to Texas public universities 


If you intend to select either the Foundation or the Foundation plus Endorsement Plan, 
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This appendix includes the informational brochure that was sent to the control group. 


Class of 2018! 


Beginning with the graduating class of 2018, Texas high school students will have the option to 
complete one of three new graduation plans: the Foundation Plan, the Foundation plus 
Endorsement Plan, or the Distinguished Level of Achievement Plan. 


With the move to these new graduation plans comes new course requirements, particularly 
with regard to mathematics. Only students who choose to complete the Distinguished Level of 
Achievement Plan or the STEM endorsement within the Foundation plus Endorsement Plan will 
be required to complete Algebra II. 


meoymantelceMlalcolaaat-lucelan-] oxelim tal-Melat-]alo[-maom uatcmC->.clom al(el ame atoto) Mel calelUr-latolamccle Uli cclnat-lnl mek) 
Niall -lswreXeColiareyar-) Mialcolgnat-]u(o)amr-] ofelUim=r-\e pKo)muat-Mo)r-latw-l ave male le) ex-lpat-lal m-]k-r-hem ol<rkXeN CAL 
the materials provided on these resource pages: 


Texas Education Agency: 
http://www.tea.state.tx.us/brochures/ 


Taksiuidel cM cola 0] 0) |Comsxel atoYo) M laliur-lahVen 
http://ipsi.utexas.edu/hb5-resources-2/ 


Education Service Center Region 13 
http://www4.esc13.net/cc/house-bill-5 
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Appendix E. Outcomes analysis results 


This appendix presents the results of the outcomes analyses. 


Table E1. Results for the multilevel model comparing Algebra II completion for 
students in treatment and control schools, 2016/17 


Variable Log odds ratio Odds ratio Standard error t ratio 


School-level variables 


Intercept 1.24 3.47 0.14 8.74** 
Treatment 0.11 1.12 0.214 0.54 
Enrollment 0.00 1.00 0.00 -1.10 

% racial/ethnic minority 0.00 1.00 0.01 0.08 

% economically disadvantaged 0.01 1.01 0.01 1.29 
% special education 0.03 1.03 0.05 0.69 

% English learner 0.01 1.01 0.01 0.50 
Algebra | end-of-course exam 0.01 1.01 0.01 1.54 
Region 2 -0.42 0.66 0.90 -0.47 
Region 3 -1.02 0.36 0.71 -1.44 
Region 4 0.414 1.12 0.43 0.26 
Region 5 -0.97 0.38 0.83 -1.17 
Region 6 -0.78 0.46 0.63 -1.23 
Region 7 -0.57 0.57 0.69 -0.82 
Region 8 -1.05 0.35 0.61 -1.72 
Region 9 -1.35 0.26 0.85 -1.58 
Region 10 -0.20 0.82 0.44 -0.46 
Region 12 -0.23 0.79 0.54 -0.43 
Region 13 -0.55 0.58 0.47 -1.18 
Region 14 -1.07 0.34 0.72 -1.47 
Region 15 -0.42 0.66 0.97 -0.43 
Region 16 -0.82 0.44 0.72 -1.15 
Region 17 -0.10 0.914 0.74 -0.13 
Region 18 -0.74 0.48 0.64 -1.16 
Region 198 -0.49 0.61 0.86 -0.57 
Region 20 0.34 1.41 0.67 0.51 
Student-level variables 

Male -0.37 0.69 0.03 -11.42** 
Asian 0.52 1.68 0.16 3.25** 
Black 0.03 1.03 0.06 0.49 
Hispanic 0.04 1.04 0.05 0.74 
Other race/ethnicity -0.11 0.90 0.114 -0.94 
Economically disadvantaged -0.17 0.84 0.04 -4.40** 
Special education -1.19 0.31 0.05 -22.85** 
English learner -0.35 0.70 0.05 -6.90** 
Algebra | end-of-course exam 0.00 1.00 0.00 40.23** 


** Significant at p < .01. 


a. Because there was just one school in region 1 and one in region 19, region 1 was combined with region 19 
for the analyses. 


Source: Authors’ calculations based on data from the Texas Education Agency Public Education Information 
Management System and State of Texas Assessments of Academic Readiness Algebra | end-of-course data. 
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Table E2. Results for the multilevel model comparing Algebra II completion for 
students in treatment and control schools with a high percentage of racial/ethnic 
minority students, 2016/17 


Variable Log odds ratio Odds ratio Standard error t ratio 


School-level variables 


Intercept 1.26 3.51 0.15 8.62* 
Treatment 0.25 1.29 0.29 0.87 
Treatment x high racial/ethnic minority -0.34 0.71 0.44 -0.78 
Enrollment 0.00 1.00 0.00 1.25 
% racial/ethnic minority 0.00 1.00 0.01 -0.18 

% economically disadvantaged 0.01 1.01 0.01 1.10 

% special education 0.03 1.03 0.05 0.67 

% English learner 0.01 1.01 0.02 0.54 
Algebra | end-of-course exam 0.01 1.01 0.01 1.44 
High-racial/ethnic minority 0.35 1.41 0.54 0.64 
Region 2 -0.44 0.65 0.914 -0.48 
Region 3 -1.14 0.32 0.77 -1.49 
Region 4 0.14 1:15 0.44 0.32 
Region 5 -1.03 0.36 0.85 -1.22 
Region 6 -0.79 0.45 0.64 -1.23 
Region 7 -0.56 0.57 0.72 -0.78 
Region 8 -1.12 0.33 0.62 -1.80 
Region 9 -1.33 0.26 0.86 -1.55 
Region 10 -0.19 0.83 0.44 -0.43 
Region 12 -0.22 0.80 0.55 -0.40 
Region 13 -0.58 0.56 0.47 -1.23 
Region 14 -1.11 0.33 0.74 -1.49 
Region 15 -0.47 0.62 0.98 -0.48 
Region 16 -0.80 0.45 0.73 1.11 
Region 17 -0.06 0.94 0.75 -0.08 
Region 18 -0.77 0.46 0.66 -1.18 
Region 198 -0.35 0.70 0.89 -0.40 
Region 20 0.35 1.42 0.68 0.514 
Student-level variables 

Male -0.37 0.69 0.03 -11.42** 
Asian 0.52 1.69 0.16 3.26** 
Black 0.03 1.03 0.06 0.49 
Hispanic 0.04 1.04 0.05 0.75 
Other race/ethnicity -0.14 0.90 0.114 -0.94 
Economically disadvantaged -0.17 0.84 0.04 —-4.40** 
Special education -1.19 0.31 0.05 -22.85** 
English learner -0.35 0.70 0.05 -6.89** 
Algebra | end-of-course exam 0.00 1.00 0.00 40.24** 


* Significant at p < .05; ** significant at p < .01. 


a. Because there was just one school in region 1 and one in region 19, region 1 was combined with region 19 
for the analyses. 


Source: Authors’ calculations based on data from the Texas Education Agency Public Education Information 
Management System and State of Texas Assessments of Academic Readiness Algebra | end-of-course data. 


Table E3. Results for the multilevel model comparing Algebra II completion for 
students in treatment and control schools with a high percentage of economically 
disadvantaged students, 2016/17 


Log odds Standard 
Variable ratio Odds ratio error t ratio 
School-level variables 
Intercept 1.22 3.39 0.14 8.60** 
Treatment 0.50 1.65 0.28 1.78 
Treatment x low income -0.91 0.40 0.43 -2.09* 
Enrollment 0.00 1.00 0.00 -1.66 
% racial/ethnic minority 0.00 1.00 0.01 0.114 
% economically disadvantaged 0.02 1.02 0.01 1.54 
% special education 0.03 1.03 0.05 0.54 
% English learner 0.01 1.01 0.01 0.86 
Algebra | end-of-course exam end-of-course exam 0.02 1.02 0.01 1.66 
Low-income 0.10 1.10 0.49 0.20 
Region 2 -0.46 0.63 0.94 -0.49 
Region 3 -1.10 0.33 0.72 -1.53 
Region 4 0.30 1.35 0.43 0.69 
Region 5 -0.85 0.43 0.82 -1.04 
Region 6 -0.84 0.43 0.65 -1.29 
Region 7 -0.52 0.60 0.69 -0.75 
Region 8 -1.14 0.32 0.61 -1.87 
Region 9 -1.49 0.23 0.85 -1.76 
Region 10 -0.13 0.88 0.45 -0.28 
Region 12 -0.10 0.90 0.56 -0.19 
Region 13 -0.51 0.60 0.47 -1.09 
Region 14 -1.17 0.34 0.73 -1.61 
Region 15 -0.50 0.61 0.96 -0.52 
Region 16 -0.69 0.50 0.72 -0.96 
Region 17 0.02 1.02 0.75 0.03 
Region 18 -0.78 0.46 0.66 -1.19 
Region 198 -0.42 0.66 0.87 -0.48 
Region 20 0.45 1.57 0.67 0.67 
Student-level variables 
Male -0.37 0.69 0.03 -11.42** 
Asian 0.52 1.69 0.16 3.26** 
Black 0.03 1.03 0.06 0.50 
Hispanic 0.04 1.04 0.05 0.76 
Other race/ethnicity -0.11 0.90 0.114 -0.93 
Economically disadvantaged -0.17 0.84 0.04 -4.41** 
Special education -1.19 0.31 0.05 —22.85** 
English learner -0.35 0.70 0.05 -6.90** 
Algebra | end-of-course exam 0.00 1.00 0.00 40.23** 


* Significant at p < .05; ** significant at p < .01. 


a. Because there was just one school in region 1 and one in region 19, region 1 was combined with region 19 
for the analyses. 


Source: Authors’ calculations based on data from the Texas Education Agency Public Education Information 
Management System and State of Texas Assessments of Academic Readiness Algebra | end-of-course data. 


Notes 


Students could drop down to the Minimum High School Program, which did not 
require Algebra II, if their school district provided written notice to the student’s parent 
or guardian explaining the benefits of the Recommended High School Program; the 
student, the student’s parent or guardian, and a school counselor or school administra- 
tor agreed that the student should be permitted to take courses under the Minimum 
High School Program; and the student met one of the following conditions: was at 
least 16 years old, had completed two credits required for graduation in each subject 
area, or had failed to be promoted to grade 10 one or more times as determined by the 
school district. 

A student may graduate under the Foundation High School Program without earning 
an endorsement if, after grade 10, the student and the student’s parent or guardian are 
notified of the benefits of graduating with an endorsement and the student’s parent 
or guardian gives written permission for the student to complete the basic 22-credit 
Foundation High School Program. 

Students could drop down to the Minimum High School Program, which did not 
require Algebra II, if their school district provided written notice to the student’s parent 
or guardian explaining the benefits of the Recommended High School Program; the 
student, the student’s parent or guardian, and a school counselor or school administra- 
tor agreed that the student should be permitted to take courses under the Minimum 
High School Program; and the student met one of the following conditions: was at 
least 16 years old, had completed two credits required for graduation in each subject 
area, or had failed to be promoted to grade 10 one or more times as determined by the 
school district. 

The endorsements include science, technology, engineering, and math; business and 
industry; public services; arts and humanities; and multidisciplinary studies. A student 
can earn a science, technology, engineering, and math endorsement by completing 
foundation and general endorsement requirements, including Algebra II; chemistry; 
physics; a coherent sequence of four or more credits in career and technical education 
that consist of at least two courses in the same career cluster, including at least one 
advanced career and technical education course, which includes any course that is 
the third or highest course in a sequence; and a coherent sequence of four credits in 
computer science selected from the approved list. 

A student may graduate under the Foundation High School Program without earning 
an endorsement if, after grade 10, the student and the student’s parent or guardian are 
notified of the benefits of graduating with an endorsement and the student’s parent 
or guardian gives written permission for the student to complete the basic 22-credit 
Foundation High School Program. 

Previously, students in the top 10 percent of their graduating class were automatically 
eligible for admission. Students who earn the science, technology, engineering, and 
math endorsement also qualify for automatic admission because they are required to 
complete Algebra II, which qualifies them for a distinguished level of achievement. 
Only admission to college and college selectivity were available in ELS. Data were not 
available in ELS for cumulative grade point average, retention, or graduation. 
Students and parents who did not wish to have their contact information publicly 
available could opt out of the directories. 
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10. 


11. 


The study team attempted to recruit more than one school in this region but did not 
receive the public information request data from districts in this region in time to 
include them. 

After the schools in the other districts were randomized, there were 57 treatment 
schools and 58 control schools. 

Students must take Algebra I in grade 9 if they did not complete it in middle school 
and then complete geometry (in that order). Although the informational materials 
were not distributed until students were in grade 10, the timing of the intervention 
is such that all students meeting the eligibility requirements in participating schools 
should have had the opportunity to complete Algebra II in grade 11, unless they were 
already off track by having failed an earlier math course. 
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